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ABSTRACT 

A low-power high-speed smart sensor system based on a large format active pixel sensor (APS) 
integrated with a programmable neural processor for space exploration missions is presented. 

The concept of building an advanced smart sensing system is demonstrated by a system-level 
microchip design that is composed with an APS sensor, a programmable neural processor, and an 
embedded microprocessor in a SOI CMOS technology. This ultra-fast smart sensor system-on-a- 
chip design mimics what is inherent in biological vision systems. Moreover, it is programmable 
and capable of performing ultra-fast machine vision processing in all levels such as image 
acquisition, image fusion, image analysis, scene interpretation, and control functions. The system 
provides about one tera-operation-per-second computing power which is a two order-of- 
magnitude increase over that of state-of-the-art microcomputers. Its high performance is due to 
massively parallel computing structures, high data throughput rates, fast learning capabilities, 
and advanced VLSI system-on-a-chip implementation. 

1. INTRODUCTION 

In this paper, an ultra-fast low-power smart vision system-on-a-chip design is proposed to 
provide effective solutions for real time machine vision applications by taking advantages of 
advances in integrated sensing/processing designs, electronic neural networks, advanced 
microprocessors, and sub-micron VLSI systems-on-a-chip (SOAC) technology. 

The proposed smart vision system mimics what is inherent in biological vision systems. 
Moreover, this vision system is programmable and capable of performing ultra-fast machine 
vision processing in all levels such as image acquisition, image fusion, image analysis, scene 
interpretation, and control functions. The system provides about one tera-operation-per-second 
computing power which is a two order-of-magnitude increase over that of state-of-the-art 
microcomputers. Its high performance is due to massively parallel computing structures, high 
data throughput rates, fast learning capabilities, and advanced VLSI system-on-a-chip 
implementation. This highly integrated smart vision system can be used for various space 
exploration missions. For example, it can be used as a smart navigation sensor that is required 
for autonomous rendezvous and docking. 

2. SMART VISION PROCESSING ALGORITHM AND ARCHITECTURE 

The human visual system provides a demonstration of a versatile and powerful vision system [1], In 
general, much of the current machine vision research concerns not only understanding the process of 
vision but also designing effective vision systems for various real-world applications [2]. 
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The proposed versatile vision system is based on a simplified model of the human visual system as 
shown in Figure 1 [3]. The ultimate design goal is to build an eye-brain machine which can automatically 
recognize, localize, and classify point, area and volume objects and phenomena in real-time. In general, 
the eye-brain machine (EBM) includes two major subsystems: the EBM Eye and the EBM Brain. The 
EBM Eye is a compact optoelectronic subsystem which integrates a wide range of different sensors with 
geometric, radiometric, and spectral parameters meeting the actual science and mission requirements. 
The EBM Brain is a high performance control and data handling subsystem, which provides computing 
resources to perform various on-board vision tasks. 




Figure 1. Smart vision processing model based a simplified human visual process flow. 



Figure 2 shows a system diagram of the proposed smart vision system-on-a-chip design. An on- 
chip row/column-parallel image flow architecture is used to connect all on-chip systems and eliminate 
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data bandwidth bottlenecks due to conventional bus architectures. The on-chip APS imager is used as the 
optical sensing array in the system. Windowed image data is fed to the on-chip neural neural processor 
under the control of a smart window handler. The on-chip neural computer is programmed to perform 
various early vision tasks in high speed. This tera-operation-per-second neural processor serves as a 
supercomputing engine for various vision processing tasks due to its massively parallel computing 
structures and its programmability. The on-chip microcomputer is used to perform command and control 
of the system operation and scene interpretation. The vision system can work with a remote host system 
through the bus interface unit. 

A system-on-a-chip implementation of this smart vision system is shown to be feasible by 
integrating the whole system into a 2-cm x 2-cm chip design in a 0.18-pm SOI CMOS technology [4] 
(see Table 1). The SOAC-based vision system design has advantages over the COTS-based design such 
as higher on-chip system integration, lOx improvement in system miniaturization, lOx lower power 
dissipation, and xlO better performance. 

Table 1: Area and power estimates for the smart vision system design in a 0.18-pm CMOS technology. 


Smart Vision Processor 

Technology: 0.18-pm SOI CMOS 


Vdd = 0.9 Volt 

On-Chip Systems 

Building Blocks 

Size (mm sq.) 

Power (mW) 

On-Chip Camera 

APS Sensor (1024x1024) 

16 

30 


Image Frame Memory (1 MBytes) 

16 

50 


Smart Window Handler 

9 

40 

On-Chip Neural Computer 

Neural Processor (1024x1024 or 
64x64x64) 

64 

200 


Synapse Memory (1 MBytes) 

16 

50 


Activity Memory (1 MBytes) 

16 

50 


Neural Processing Controller 


40 

On-Chip Microcomputer 

Microprocessor 

45 

200@ 300 MHz 


Memory (32 Mbytes) 

144 

100 


Bus Interface and Control 

16 

80 


Total: 

335 

790 


3. ON-CHIP ACTIVE PIXEL SENSOR DESIGN 

High performance CMOS active pixel sensor technology has been developed by NASA's Jet 
Propulsion Laboratory for electronic image capture [5]. A 1024x1024 CMOS APS prototype 
chip using 0.5 pm n-well process was designed and characterized at JPL. It contains a 
1024x1 024-photogate array and 1024 parallel 10-bit singles-slope ADC. Testing results show 
that the large format APS with small feature size is capable of excellent imaging performance. A 
reusable on-chip APS imager based on this prototype APS chip design has been under 
development in a 0.18 pm SOI CMOS technology [6]. 

The on-chip APS imager is based on a row/column/window-parallel data flow architecture. It 
includes an APS active sensor, a smart window handler, and a parallel-I/O image frame memory. 
The smart image window handler is designed for the row/column/window-parallel interface 
between the APS imager and the neural processor to achieve an ultra-fast frame time up to 1000 
frames per second. 
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4. ON-CHIP PROGRAMMABLE NEURAL PROCESSOR DESIGN 


An on-chip programmable neural processor design is used as a parallel processor to perform various 
vision functions at very high speed. Incorporating the neural computer into the proposed vision system 
offers orders-of-magnitude computing performance enhancements for real-time vision tasks. 

The programmable neural processor is based on optimization cellular neural network (OCNN) [7], 
The OCNN is an improved version of the Cellular Neural Networks (CNN) [8], The CNN has been 
proved to be universal as the Turing machine. The OCNN keeps this highly desired programmability and 
become a versatile vision processor. Many OCNN functions have been verified via system simulation. 
These functions include morphological operations, feature extraction, motion detection, path tracking, 
collision avoidance, etc. The OCNN architecture is a multi-dimensional array of mainly identical cells, 
which are dynamic systems with continuous state variables and locally connected with their local cells 
within a finite radius. A prototype chip of the OCNN was designed and tested [7], An on-chip OCNN 
neural processor based on this prototype OCNN chip has been under development for the proposed smart 
vision system. The parallel column/row/window readout capability is also integrated with the OCNN to 
accommodate the applications that require high-speed parallel image I/O. 

5. CONCLUSION 

A high-speed low-power smart vision system-on-a-chip design is proposed to provide effective 
solutions for space exploration missions. A system-on-a-chip implementation of this smart vision 
system is shown to be feasible by integrating the whole system into a 2-cm x 2-cm chip design in a 
0.18-pm CMOS technology. An on-chip row/column-parallel data flow architecture is used to connect 
all on-chip systems and eliminate data bandwidth bottlenecks due to conventional bus architectures. This 
highly integrated smart vision system-on-a-chip design can be used on various space exploration missions 
and other industrial or commercial vision applications. 
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